The present disclosure relates generally to machine learning, and more particularly, to methods for training of deep neural network models to perform structured classification tasks.
Neural networks are simplified models of the brain comprising units associated by weighted connections. A weight on a connection reveals the strength of the connection.
Neural networks have demonstrated an ability to learn such skills as face recognition, reading, and the detection of simple grammatical structure. More particularly, neural networks can be considered to be models defining a multivariate function or a distribution over a set of discrete classes. In some instances, neural network models can be associated with a particular learning method or learning rule.
The use of neural networks for machine learning tasks, including acoustic modeling for speech recognition, is well known. The ability to train increasingly deep networks has been due, in part, to the development of pre-training algorithms and forms of random initialization, as well as the availability of faster computers.